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ABSTRACT 


This thesis describes the use of a prototype Tactile Situational Awareness 
System (TSAS) as an approach to aid pilot performance following simulated laser 
blindness modeled during a virtual approach in an SH-60 helicopter. Situational 
awareness and spatial awareness remain critical factors for successful control of 
manned aircraft. Helicopters and fixed winged aircraft pilots react to spatial 
orientation challenges during take-off, and landing phases of flight. U.S. and 
NATO aircraft pilot surveys examined the human machine interaction and 
revealed degraded vision as an important human factor contributing to mishaps 
or near mishaps. Vision was identified as an information chokepoint limiting 
command and control of the aircraft. Fortunately, vision can be augmented with 
an available technology called “haptics” during restricted or limited human vision. 
Therefore, an experiment using X-Plane output for haptics-generated input from 
a torso-worn TSAS was developed. Participants received haptic cues during 
runway approaches after experiencing simulated loss of vision. Participant 
performance after simulated laser blinding with and without the TSAS compared 
time advantage and navigation accuracy. Simulator performance data indicated 
pilots using TSAS following simulated laser blindness responded to haptic cues, 
had more time to prevent the aircraft from obtaining an unsafe pitch or roll 
condition, and could position the aircraft closer to the landing zone. 
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I. INTRODUCTION 


A. BACKGROUND 

In aviation, there is a concern that laser blinding could cause pilots to 
crash because of an inability to see. The pilot has to know how to interact and 
manage time with the aircraft, crewmembers, weather conditions, other aircraft, 
birds and personal flight equipment. As for the flow of tasks, the pilot interacts 
with the aircraft, the environment, and the airspace. The pilot is constantly 
scanning instruments, manipulating controls, and listening for alarms. For each 
of the three areas, vision is the source of information the pilot used to gather 
necessary information for stabilized flight. A proposed solution to the laser 
problem is to test pilots wearing the Tactile Situational Awareness System 
(TSAS) as an aid to continue controlled flight after losing visual references 
outside the cockpit and to flight instruments. 

The hope is that, using the TSAS, the pilot will be able to fly safely to a 
landing zone, saving lives and aircraft. According to Brooks and Madden of CNN 
(2005), the threat from potential laser blindness is increasing. It occurs when a 
pilot is on final approach to an airfield and reports a laser light pointed at the 
aircraft cockpit. Fortunately, no cases of laser activity have resulted in an 
aviation accident to date. However, should the laser have the appropriate power 
to blind a pilot, such a situation could be catastrophic. TSAS could be useful 
during a blinded pilot situation to cue the pilot to maintain aircraft orientation and 
fly safely to a landing zone. 

B. MOTIVATION 

According to Nakagawara (2003), pilots use vision for over 90 percent of 
input cues in command and control of an aircraft. Visual cues allow quick 
updates for hand-to-eye coordination in the cockpit and help pilots perform their 
jobs with ease. Visual information has become an information single point of 
failure for pilots. Unfortunately, there is not a fielded device to substitute for the 
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required high scan rates, or to allow a pilot to receive information and monitor 
critical flight instruments and alarms in the cockpit in the event vision is lost. 

Procedurally, military manuals such as Naval Air Training and Operating 
Procedures Standardization (NATOPS) manuals and the Federal Aviation 
Administration Flight Air Rules and Aeronautical Information Manual (FAA 
FAR/AIM) do not cover or adequately address how to handle this situation— 
except to gain altitude, engage the autopilot if available and wait for vision to 
return and file a pilot report. 

Currently, primary flight training programs do not incorporate the use of 
the TSAS for pilot training. However, the technology and concept exist. The 
TSAS affords pilots a tool to aviate when experiencing reduced visibility. 

C. MODELING TOOLS TO SIMULATE FLIGHT ENVIRONMENT 

The MOVES Lab contained a simulated flight-training device. By feeding 
data from the flight training device computer to the TSAS and developing an 
appropriate software interface program, simulated flight with the TSAS was 
feasible. This approach provided a mechanism to record flight data during a 
simulated laser-blinding emergency. Collected data resulted in measured 
performance of Participants with the TSAS during simulated vision loss to the 
pilot in a virtual flight environment. 

D. OBJECTIVE AND HYPOTHESES 

The objective was to record and compare participant performance level 
with and without the TSAS following a simulated laser-blinding event, using a 
virtual helicopter approach and show the value added using the TSAS. 
Furthermore, the military and the aviation industry gains another set of data to 
strengthen the case for developing more cockpit instrumentation systems based 
on the TSAS in the future. 

The researcher explored three hypotheses concerning the use of the 

TSAS in this thesis. First, the TSAS augments degraded or lost vision during an 

approach. Second, the TSAS can provide feedback following laser blindness 
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and inform the pilot how to navigate the aircraft toward a specified landing zone 
in the absence of visual cues. Third, the TSAS can function as a haptic 
instrument for maintaining controlled flight in a virtual environment. Finally, what, 
if any, are the new negative consequences added using the TSAS, which could 
affect safety of flight? 

E. THESIS OUTLINE 

The layout of this thesis is as follows: Chapter II provides background and 
literature review. Chapter III describes the methods, procedures, and 
experimental setup. Chapter IV presents results based on data retrieved from X- 
Plane for each participant and the results from participant post-questionnaire. 
Chapter V discusses the conclusions, reports if the TSAS was an effective haptic 
flight instrument for flight, describes the limitations of the thesis, and provides a 
list of potential paths for continued future work to explore other potential 
applications with the TSAS. 
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II. LITERATURE REVIEW 


A. HAPTIC SENSES 

Why consider a haptic sensor display? Before addressing the design, a 
brief background on the interaction of the human body and haptics is necessary. 
According to Gibson (1966), the sensibility of the individual interacting with the 
world adjacent to the body occurs through the response of nerve and pressure 
sensors on and beneath the body’s largest organ, the skin. Therefore, the 
largest organ the skin functions as one large haptic system. According to 
Geldard (1972), the skin is receptive to stimuli including changes in temperature, 
pain, and pressure. In this thesis, the focus is on the flight environment in which 
a rubberized Body Glove™ vest lined with haptic input devices interacts with the 
body, providing cueing information about the state of the aircraft in the absence 
of sight. Further research by Hendrix and Durfee (2003) explored haptics with 
Human Computer Interfaces (HCI) as an advantage over visual interfaces. 
Hendrix and Durfee used Fitt’s law to model HCI accuracy and movement. 

The interaction medium for this haptic system is the nervous system just 
below the epidermis and dermis skin layers. The tiny hairs covering the body 
pass through the epidermis and dermis and connect to nerve endings. The 
human tactual system provides a proximity sense and the feeling of a stick or 
prick to the body. The Ruffini corpuscles are the nerve receptacles beneath the 
skin capable of detecting changes in temperature. The Merkel receptors sense 
external pressure changes and respond to frequencies from 0.3 Hz to 3 Hz. 
Velocity fluctuations depend on the Meissner corpuscles under the skin and have 
a sensitivity range from 3-40 Hz. The Pacinian corpuscles from Anatomy of the 
Skin (2006) (see Figure 1) classified as the largest skin receptors fit the desired 
characteristic needed to conduct the experiment. Research by Engineering 
Acoustics Incorporated state the Pacinian corpuscles are sensitive receptors and 
respond to input changes primarily in the 200Hz-350Hz zone, allowing high 
sensitivity to acceleration and vibration from sensor devices like the C-2 tactor 
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placed next to the skin. An added benefit of the Pacinian Corpuscles found by 
Klatzky and Lederman (2002) is that they are rapid adapting receptors within sub 
cutaneous tissue and the effect of stimuli to pressure and vibration decays 
rapidly. 
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Figure 1. View of Pacinian Corpuscle (From Anatomy of the Skin, 2006) 


The TSAS lined with C-2 tactor devices creates a haptic instrument to 
stimulate the haptic senses. Haptic systems consist of external apparatus by 
which the individual receives information concerning the state of a platform in an 
environment through interactions with the body. By using haptics, the pilot stays 
aware of the information available from cockpit instruments without having to rely 
solely on vision. Haptics also reinforces awareness of the aircraft attitude 
because the torso is normally buckled into the aircraft forming perpendicular 
alignment to the normal of the aircraft, whereas the pilot’s head may tilt (Liggett 
2002 from Hasbrook and Rasmussen, 1973) skewing the relationship of the 
aircraft to the horizon. 
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B. LITERATURE REVIEW 


Haptics is one of the first senses that humans encounter during 
development (Gibson, 1966). Booher (2003) defined haptic vision as using a 
sense of touch to maintain situational awareness of one’s surroundings. The 
application of haptics or tactile devices for flight purposes began in the mid- 
1990s with proof of concept performed in 1995 in a Cessna 172 (Rupert, 1997). 
According to Rupert, he designed the haptic piloting device and named it the 
Tactile Situational Awareness System (TSAS). Naval Air Base Patuxent River 
conducted military test of the TSAS using a T-34C test aircraft in 1995 (Rupert, 
1997). Several other TSAS experiments have followed the pioneering work of 
Dr. Rupert. The Netherlands used the TSAS in a study to reduce side drift while 
training pilots in the use of night vision goggles (NVG) during flight. Researchers 
Erp, Veltman, Veen, and Oving (2002) of the Netherlands reported students in 
the Royal Dutch Airlines Flight Academy experienced improved night flights 
during normal and NVG flights while wearing a torso TSAS piloting aid. 

The first pneumatically-activated version of the TSAS resembled an 
aviation harness (see Figure 2). Technology progression allowed a transition 
from pneumatically-driven tactors to vibro-tactile devices using on board aircraft 
and backup electrical power (McGrath, Estrada, Braithwaite, Raj, & Rupert, 
2004). The pneumatic version required bulky connections and an air pump to 
supply the air for the tactors inside the vest. 



Figure 2. Pneumatic style TSAS for Military Aircraft (After McGrath et al., 2004) 
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Non-aviation variants of the TSAS are in use. Divers used haptics to 
navigate through murky water to reach waypoints under the sea and along 
coastal regions (Erp, Veen, Janseen, & Dobbins, 2005). An Astronaut 
demonstrated the TSAS ability to assist with maintaining orientation with the 
International Space Station (ISS) while performing extra-vehicular activities (Erp 
and Venn, 2003). Research by Cardin, Vexo and Thalman (2006) support the 
use of the TSAS as a flight station alarm remote indicator. Cues from the TSAS 
alerted the off-duty pilot during long transatlantic or transpacific flights if an alarm 
sounded or the other pilot at the controls needed assistance. 

How can the TSAS help in aviation? One possible goal is to help reduce 
CLASS A mishaps. According to Operational Naval Instruction ( OPNAVINST ) 
5100.23G, CLASS A mishaps, also known as Category 1 and catastrophic 
accidents, are those resulting in a total of one million dollars or more in material 
property damage, a fatality, or permanent total disability. Mishaps having 
degraded vision as a factor fall into the spatial disorientation category. 
Unfortunately, spatial disorientation, when reported in aircraft mishap 
investigations, receives a classification of pilot error as the causal factor 
(Shender, 2004). General Aviation pilots have a higher number of Class A 
mishaps. From 1983 to 2007, the National Transportation Safety Board (AOPA 
2008) recorded 744 fatal accidents for the General Aviation Community. The 
need for reducing the number of Class A mishaps has reached the congressional 
level, and a goal set by the Secretary of Defense under Department of the Navy 
Objectives for 2006 was a reduction in the baseline number of Class A mishaps 
by 75 percent before the end of fiscal year (FY) 2008. Furthermore, the number 
of acceptable Department of Defense losses is zero according to Mr. Christopher 
Bolkcom ( a specialist in national defense, foreign affairs, defense and trade 
division, congressional research service), during testimony to the 108*^^ Congress 
February 2004 on aviation safety initiatives. Increased aircrew safety and the 
prevention of aircraft loss caused by human factors are potential benefits 
obtained using the TSAS as an instrument. 


8 



According to Nakagawara et al (2003), the Department of Transportation 
and the FAA in 2003 conducted a critical flight zone test in a controlled simulated 
environment to test the effects of increased laser illumination on terminal 
operations with eye safe levels of power between 0.5 and 50 Microwatts per 
square centimeter (pW/cm^). The FAA concluded that within 30 seconds, a pilot 
with exposure to a laser regained vision sufficiently to perform flight operations. 
However, above 50 Microwatts, the effects were serious. Thus, there is a need 
for the functionality of the TSAS to augment vision and allow the pilot to maintain 
flight until safely over a suitable landing field. Ultimately, the TSAS adds a haptic 
interface for the pilot to help minimize vision-induced mishaps in planes and 
helicopters. Reducing visual instrument scans inside the cockpit during high 
workload flight operations allow more attention outside the cockpit. 

Several operational and psychophysiological surveys collected on USAF, 
USN, USA, Hellenic Air Force and the United Kingdom pilots and consolidated by 
the North Atlantic Treaty Organization Research and Technology Organization 
(2005) support the impact of spatial disorientation from the lack of visual cues. 
All aviation forces within the U.S. services and the surveyed North Atlantic Treaty 
Organization (NATO) countries listed visual effects such as blending of earth and 
sky, not detecting side drift for helicopters, the leans, misleading attitude cues 
from instruments, loss of horizon during instrument meteorological conditions, 
and distractions because of task load as factors contributing to safety of flight. 

The TSAS has not become a common application in cockpits. Reports by 
pilot subjects exposed to older TSAS models demanded better tactor technology, 
better integration and miniaturization of components. Insufficient cooling causing 
discomfort while wearing the TSAS with flight gear limited the use of the TSAS in 
one study (McGrath, Estrada, Braithwaite, Raj and Rupert 2004). 
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III. METHODS AND PROCEDURES 


A. PARTICIPANTS 

The Institutional Review Board (IRB) granted an IRB approval letter for the 
proposed research (Appendix D). Nine Participants volunteered after responding 
to student body email, MOVES Brown Bag lectures, and announcements from 
previous instructors. 

B. PRE-FLIGHT QUESTIONNAIRE 

The Pre-Flight Questionnaire (Appendix B) contained questions designed 
to determine if a Participant was susceptible to simulator sickness and to help 
collect level of flight experience. One question asked the Participant about the 
TSAS and possible training benefits. This question was designed to determine if 
knowledge of such a device was reaching aviators. All Participants received a 
summary of the experiment and signed a Participant consent form outlining their 
rights to participate and the request to stop the experiment at any time. Time to 
train prior to the scenario was determined from data collected on the 
questionnaire. Non-pilots required more familiarization with the controls, 
whereas experienced pilots did not. Both groups required the same amount of 
familiarization with the TSAS because none of the Participants had any prior 
experience using such a device. 

C. APPROACH TECHNIQUES 

Approach occurs when the helicopter transitions from level forward flight, 
descends to a hover over a landing zone, and lands. The helicopter approach 
process has four segments the pilots must master: approach angle, rate of 
closure, power control, and landing. 

Approach angle is typically eight to ten degrees down pitch from the 
horizon and appears steep to fixed wing pilots who normally are familiar with a 
standard three-degree approach on glide slope common with most instrument 
approaches. The procedure commences by establishing an eight-degree to ten- 
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degree approach angle, which can be visual or assisted by runway lighting 
systems. The pilot maintains the approach angle by performing a pitch 
adjustment for the angle of descent and establishing a view of the field or landing 
zone (LZ) through the windshield. Constant speed is not a factor since 
helicopters do not stall at low speeds as in fixed wing aircraft. Helicopter pilots 
monitor groundspeed and the landing skids alignment with the LZ while on 
approach to prevent rollover. By manipulating the collective, they control sink 
rate read from the vertical airspeed indicator (VSI) or by visually judging rate of 
descent. The landing skids are the landing gear or rails under a helicopter on 
which it lands. The VSI provides the pilot a display of vertical rate in feet per 
minute as the aircraft climbs or descends. Approach operations place the 
helicopter close to the “back side of the power curve,” which is the point if 
entered yields no response from the aircraft regardless of power adjustments via 
the collective. The speed of the approach starts between 40 and 50 knots 
airspeed and terminates in a hover. For inexperienced pilots, defined as less 
than 20 hours in type aircraft, the hover may occur early before arriving over the 
intended landing point. Because the angle of approach shifts from a ten-degree 
approach to a pitch up or flared attitude, pilots push forward on the cyclic to 
continue the approach and to avoid the chance of the helicopter’s tail rotor 
striking the ground. Finally, in termination, the pilot completes the approach by 
ensuring the helicopter is in a level hover or slightly forward motion with the 
landing skids. Alignment of the skids with the LZ is necessary to prevent a 
rollover should the helicopter approach too quickly and is not aligned with the 
direction of travel. Voice communications exchanged with approach control prior 
to termination phase provide vectors to the LZ. Once the approach commences, 
the pilot switches to tower control, if available, and then to ground control to 
determine a position to park or refuel. 

D. APPARATUS 

The research TSAS (see Figure 3) was loaned to the Naval Postgraduate 
School by the University of Central Florida in Orlando after an experiment to 


12 



enhance training in a virtual environment for infantrymen during a room clearing 
procedure (Fowlkes, Washburn, Eitelman, Daly, & Cohn, 2006). A quarter 
placed next to the bottom center C-2 tactor (see Figure 3) illustrates the relative 
size of each tactor. 



Figure 3. TSAS Courtesy of University of Central Florida (From Brown, 2007) 

Engineering Acoustics Incorporated (EAI) manufactured the C-2 tactors 
(see Figure 4) used as the cueing devices for the pilot. 



Figure 4. C-2 Vibro-tactor by Engineering Acoustics, INC (enlarged photo). 

The TSAS consisted of a vest garment and with leg straps. The tactors 
lining the TSAS vibrated according to programmed flight parameters based on 
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selected simulated flight environment. The wearer, who is taught the relationship 
of the haptic stimuli with the flight instruments, then performs a control input to 
continue flight. 

E. PROCEDURE 

A time of forty-five minutes was determined as the requirement for each 
participant to complete the experiment using the MOVES CAVE Flight simulator. 
Phase 1 provided training to fly the simulator for inexperienced and experienced 
pilots without the TSAS. Next, all Participants were familiarized with the 
operation of the TSAS during a fitting session and to experience the cues 
received from the TSAS within a normal flight environment. Roll cue 
familiarization required the Participants to place the aircraft in a roll that 
exceeded positive or negative seven degrees and recognize the corresponding 
TSAS correction cues. Roll and pitch values selected were based upon staying 
within normal maneuvering flight envelop as described in approach techniques 
and to ensure positive flight control inputs from pilot was required to maintain 
safe flight. During pitch familiarization, the Participant received feedback cues if 
a nose high pitch of 20 degrees occurred or a dose down pitch exceeding 
negative eight degrees occurred. Anti-torque or rudder pedal cues were 
provided to the participant via upper thigh cues if the aircraft heading changed by 
more than one degree. During Phase 2, the Participant wore the TSAS in a 
degraded vision state. This phase measured Participant performance while 
navigating toward the landing zone and during a TSAS assisted approach. The 
scenarios required the pilot to experience a loss of vision state and be able to 
shift from visual cues to haptic cues with no warning. The standard for 
successful performance was the pilot receiving and understanding cues to 
navigate to the landing zone without crashing. 

F. SCENARIO DESCRIPTION AND TASKS 

The flight plan was a business trip to visit the Jack Daniel’s Distillery near 
Tullahoma, Tennessee. Weather was clear and this was a visual flight rules 
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landing. The visual approach for Tullahoma Tennessee Regional Airport (THA) 
Runway number Three-Six (36) was the flight environment selected (see Figure 
5). Runway 36 corresponds to the closest magnetic heading of 360 degrees. 
The simulated location was ideal because all the terrain for the landing field from 
X-Plane synchronized with Delta-3D graphics that allowed displaying the terrain 
in the MOVES Institute CAVE. Tullahoma airport elevation was 1083 feet mean 
sea level. The aircraft contained fuel to conduct one wave off and attempt 
another approach to the same airfield but not enough fuel to fly to a divert airport. 
To keep Participants from using instruments and to limit the transition from visual 
to haptics, the researcher informed all Participants the aircraft was operational 
and they were flying solely on visual references. All visual approaches 
commenced four nautical miles from the landing zone. 
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Once the Participant was one nautical mile south of the landing zone, the 
researcher virtually removed the pilot’s vision by pressing the “no-image” button. 
The geographic location south of the landing zone received the name “incident 
point.” Incident approach point placed the aircraft 800 feet above ground level or 
1883 feet mean sea level. Incident point was determined based on the aircraft’s 
approach altitude and heading. When reaching Incident Point, the assumed 
laser event with sufficient power blinded the pilot. Blinding was based on the 
assumption that protective lenses or helmet visor were not worn. Once this 
incident occurred, the user continued flight without visual cues. The researcher 
disabled X-Plane voice communications in each phase. The pilot, with or without 
the TSAS, was instructed by the researcher to continue and attempt to reach a 
safe landing zone using only memory of visual references prior to the incident. 
With the TSAS, haptic cues corresponding to aircraft control input corrections 
directed the user to the field. In the virtual environment used, no motion was 
available to the pilot to sense aircraft forces or the forces of gravity and 
maneuvers. To add some virtual immersion, the available audio system in the 
MOVES CAVE amplified simulated aircraft sound effects created by X-Plane. 

Half of the Participants used TSAS first and half experienced the “No 
TSAS” condition first. Once the instruction and familiarization phase for the 
training device was over, the Participants tried on the TSAS around the torso and 
received instruction on the cues the TSAS would provide during flight. The 
tactors received input signals based on X-Plane data activated by pitch, roll, and 
heading set points programmed into the code in Appendix A. The input signal 
drove the appropriate C-2 tactor and provided cueing feedback from the cockpit 
instruments to the pilot. The cues were intended to help the pilot maintain 
heading, and normal pitch and roll attitude, while approaching a landing zone. 
The Participants performed the following tasks: 

1. Maintain control of the aircraft by maintaining altitude and keeping 
the horizon level. 

2. Continue to landing zone using TSAS inputs. 
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3. Perform a haptic visual approach using the heading cues to landing 
field with the TSAS. 

4. Provide feedback after Phase 2 on post event questionnaire. 

The roll cues corresponded to signals from the upper left and upper right 
front tactors and indicated left cyclic or right cyclic inputs, respectively. Tactors 
positioned at the left and right upper back corresponded to aircraft pitch 
corrections based on limits chosen in Appendix A. The TSAS contained two 
connections that provided the user with cues from Velcro strapped leg tractors on 
each leg. When the user received a cue, the user applied either left or right anti¬ 
torque pedal inputs to steer back and maintain the aircraft on the desired heading 
toward the landing zone. The two tactors on the lower front were intended for 
collective or power inputs but not used. Two tactors on the lower back of vest 
were in place as spares in case one of the other tactors malfunctioned. The 
small number of tactors used enabled Participants to understand and remember 
the meaning of the tactor inputs necessary to perform the mission. 

G. RADIAL OR HEADING KEEPING DURING APPROACH 

Radial or heading keeping normally involves the use of a single navigation 
receiver dialed up to a radial on a navigation aid or magnetic fix by which the pilot 
can locate an airport or a landing zone. The process requires the pilot to make 
adjustments to keep the aircraft on the radial and monitor aircraft position until 
safely landing. The process consisted of a constant feedback information loop 
designed by the researcher especially for this thesis (see Figure 6). 
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Figure 6. Radial Keeping to Landing Decision Flow Path 


From the radial keeping flow path, signals programmed into the TSAS 
interface box provided the participant with a cue indicating whether the aircraft 
flew left or right of the desired heading or navigation radial. 

H. HUMAN ABILITIES REQUIREMENTS (HARS) 

In order to illustrate the dependency on vision and why a TSAS could 
prove beneficial, a task analysis and human abilities requirement (HARS) 
analysis was conducted. The requirements and frequency of cognitive, motor 
and perceptual tasks required to conduct a visual approach in a virtual simulator 
environment appear in Table 1. 
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Deductive Reasoning | 
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Speed of Closure | 
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Perceptual Speed | 

Spatial Orientation | 

Visualization | 
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Table 1. HARS Table with Frequently Linked Items Identified by a Dot 
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Next, the subtask in the left column of Table 2 display the score based on 
counting the dots required for the task from Table 1. The ranking of the subtasks 
coincided with all aviation training, stressing the need to aviate first, navigate, 
and then communicate with crew and outside agencies such as air traffic control, 
approach, tower, and ground control. 
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I. 


DESCRIPTION OF EXPERIMENTAL SETUP 


1. MOVES CAVE Simulator Setup 

The MOVES lab flight simulator was suitable for the needs of this 
experimental evaluation. The simulator uses a commercially available software 
package called X-Plane, which was licensed to NPS. The FAA accepted X- 
Plane in 2002 as a simulation tool certified to meet Part 61:4(a) and Part 141.41 
requirements for a Level 2 Flight Training Device. The MOVES CAVE Flight 
Simulator (see Figure 7) system required system startup, data recording, and 
system shutdown procedures. 



HOST FUGHT MOOEL 


Figure 7. MOVES CAVE Components Photo Courtesy Delta-3D.org 

During startup, the Common Image Generator Interface or (IG), an Open 
Source game engine, provided the visual graphics for the viewer. The IG 
graphics projected video on three 7-square-foot screens to create a sense of 
immersion for the simulated flight environment. The researcher monitored the 
same images using a remote triple-head Liquid Crystal Display (LCD) to monitor 
the flight with and without the TSAS. The Flost computer. Flight Model computer, 
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and Audio System pictured in Figure 7 controlled operation of the CAVE. The 
Host system supported an instrument gauge operation using a driver package 
called “TRC Custom” developed by Erik Johnson. The Flight Model computer 
ran X-Plane version 7.5. The audio system consisted of an Onkyo Stereo 
receiver, a Klipsch pre-amp, Buttkicker™ amplifier and GENELEC™ speakers. 

The flight instrument panel consisted of the basic instruments for flight in a 
helicopter. They were the compass, altimeter, airspeed indicator, horizontal 
situation indicator (HSI), vertical speed indicator (VSI), single navigational aid, 
and a two-minute turn indicator (see Figure 8). 



Figure 8. MOVES Flight Research CAVE 


Once powered up, the researcher used X-Plane data menus to select the 
parameters for recording. This required using the X-Plane Settings and Data 
Input/Output menu accessed from the drop-down menu bar. The process 
required choosing which computer to serve as the receiving unit and determining 
whether another computer would duplicate the data based on internet protocol 
(IP) address. Altitude, Heading, Latitude, Longitude, Pitch, and Roll parameters 
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were selected. The update rate of the data was adjusted to minimize the amount 
of data recorded while insuring a minimum 15 frames per second rate to maintain 
a quality virtual environment — and reduce the chance of simulator-induced 
sickness. Analyzing tool eXaminer version 1.0 was used to review the type of 
data collected from the X-Plane computer User Datagram Protocol internet 
protocol (UDP IP) address and the port number specified (see Figure 9) prior to 
recording the output. 



Figure 9. X-Plane eXaminer v1.0 Example Setup 



Figure 10. X-Plane Flight Recorder with IP and Port Settings 
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The use of X-Plane Flight Recorder v6.20 Beta 2 (see Figure 10) shows a 
typical setup using the IP Address and port number for the sending computer 
between the host computer and a second computer. This setup became the 
training setup with the TSAS and recorded the data with text files sent to data.out 
file extension for archives. To preserve privacy, each data.out file received a file 
name linked to each participant number. The data contained in the file consisted 
of parameters marked with an 'x' from the X-Plane Data Input and Output 
selection menu and the Data sub-menu. The specific index items selected were 
00 for the frame rate, 07 for the joystick, 16 for the pitch, roll, and heading values, 
and 18 for latitude and longitude during flight. The GPS parameter 89 
synchronized X-Plane video with the MOVES Institute Virtual Tullahoma Delta- 
3D scenery. 

The next window used under Data Input and Output was the X-Plane Inet 
sub-menu for data internet UDP protocol address, port assignment, and data 
send rate adjustable in frames per second. Unfortunately, this window’s font was 
too small to display within the text of this thesis. The last index item on the sub¬ 
menu page selected matched the machine IP address of the data receiver 
machine. This machine contained the code developed to operate the TSAS and 
perform the initial calibration with the participant. 

Since Participants for the experiment came from the Naval Postgraduate 
School population, time to train each participant to fly was limited and not the 
primary focus for the research with the TSAS. Therefore, the nulizone of the 
pitch, roll, and heading from the Flight Line™ stick adjustment reduced the high 
sensitivity of the controls and made the aircraft more stable (see Figure 11) 
during the flight. 


25 




Figure 11. X-Plane Nulizone Adjustment 

Shutdown consisted of securing the IG System, Instrument Panel, Audio 
system. Host and Flight Model computer. Projection system and remote monitor 
last. 


2. Battery Pack Power Supply 

Throughout the design phase of the experiment, one safety concern was 
to remove the requirement for the participant to have commercial power used 
during the experiment. A battery back provided added mobility without the 
limitation of an electrical extension cord. Determining the amount of battery 
power required a power consumption test on the TSAS. The solution was a 12V 
DC portable battery source. The test indicated a participant could use the battery 
pack for two hours of continuous operation before the batteries needed replacing. 

Using the battery pack for the TSAS during test and calibration in a static 
environment confirmed full operation while worn with tactors activated. Heat 
from the interface box using commercial power or a battery pack created a minor 
safety consideration. Installing a 2.36-inch fan cover satisfied a requirements for 
participant safety and the protection of the interface fan (see Figure 12). 
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Figure 12. Tactor Interface Box with Fan Cover Installed 

Each C-2 tactor received a unique index parameter assignment to cue the 
participant based on changes in data received from X-plane. Front and rear 
drawings of where the C-2 tactors were located on the participant’s torso and the 
corresponding data cue received appear in Figures 13 and 14. 
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Figure 13. 


C-2 Tactor Approximate Front Physical Location 
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Figure 14. C-2 Tactor Approximate Back Physical Location 


J. CODE DEVELOPMENT AND CHALLENGES 

Code from the work performed by Fowlkes et al., became a starting point. 
Initial operation of the TSAS, interface box, and each tactor required the EAI 
Demo program. This program confirmed that TSAS responded manually without 
data from X-Plane. Next, the program X-Plane Tactor read data over UDP from 
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the Flight Model machine running X-Plane to the host machine used to drive 
each TSAS tactor. Duplicate data and command signals for the TSAS traveled 
from the host computer to the TSAS user through a serial port. The serial port 
was simpler to implement and excluded the USB port option on the interface 
panel. The interface allowed assignment of a listening port and a communication 
port on the receiving computer. This mouse activated graphical user interface 
(GUI) started data collection — provided the network was up and running and the 
firewall enabled UDP data. 

1. Data Monitoring 

The remote station allowed the researcher to monitor the received data in 
two ways. The first was a window to show the packets received and the second 
was a program named Portman that recorded and displayed hexadecimal and 
binary data received. Initially, data received did not result in a response to the 
assigned tactors on the TSAS. By observing the incoming data, Portman 
revealed the hexadecimal value-reading format needed to shift the data reading 
order from Big Endian format to Little Endian format or from left to right as 
addressed in a text by Null and Lobin (2003). 

2. Technical Challenges 

Driving each tactor from the data received was a challenge because the 
values of the X-Plane joystick index ranged from a unit less negative one to 
positive one for maximum deflection. During simulated flight, joystick deflection 
stayed between negative decimal five and positive decimal five for normal flight. 
Precision of data received from X-Plane was excellent because, when printed out 
from the program, the output was 17 significant digits more than required for 
recording approach deviations. 

Cueing each tactor to provide the necessary correction input was 

challenging because each tactor needed to be snug against the participant to feel 

the vibration. Additionally the manufacturer, EAI, recommended an ideal C-2 

duty cycle of 10 percent on and 90 percent off. Incrementing the amount of time 
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the tactor vibrated showed the time needed was between 50 milliseconds and 
300 milliseconds to ensure proper recognition of tactor input. Providing cues 
from the TSAS for the anti-torque pedals was achievable by applying a tactor 
input to each leg for left or right anti-torque pedal input. The input cued the 
participant to turn toward a particular heading, or was used to prevent yaw 
tendencies when power was increased or decreased. 

Providing the user with cues to counter extreme pitch and roll required 
reading in data from X-Plane index number 16 and comparing the value received 
with a tolerance. The port number, communication port, and menu interfaces 
used to toggle the anti-torque, pitch and roll data from X-Plane to the TSAS 
appear in Figure 15. The interface prevented overdriving the tactors if X-Plane 
sent data to TSAS without a user. 



Figure 15. X-Plane to Tactor User Interface 

Initial calibration, training, and flying (see Figures 16, 17 and 18) 
presented technical challenges to achieving the correct virtual refresh rate, while 
limiting an overload of data recorded for plots and analysis. The remedy required 
resetting the refresh rate back to 15 frames per second after each reboot of the 
host and flight model machines. Mapping of X-Plane to the projection screens 
was possible by using Delta-3D software developed at NPS by the MOVES 
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Institute Team. The airfield in Delta-3D matched runways in X-Plane for 
Tullahoma Regional Airport visual flight rules (VFR) final approach. 


Figure 16. 



Calibration Wearing TSAS and Leg Straps (From Sadagic, 2007) 
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Figure 17. TSAS Training in MOVES CAVE (From Sadagic, 2007) 



Flying Simulated SFI-60 in MOVES CAVE (From Sadagic, 2007) 
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Figure 18. 



If a factor was non-responsive on the calibration test, a spare was 
available as a replacement. Barometric pressure adjustments ensured approach 
altitudes measured in feet mean sea level (MSL) corresponded to the correct 
height assigned. 
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IV. RESULTS 


A. PARTICIPANTS 

The participant sample consisted of eight males and one female. The skill 
level of the nine Participants consisted of five helicopter pilots, one fixed wing 
general aviation pilot and three non-pilots. Flight experience among the 
helicopter pilots averaged 1,590 hours flight time, as noted in Table 3 created 
from the Pre-Flight questionnaire. Of the nine Participants recruited, six 
completed all test scenarios with and without the TSAS. Two non-pilots and one 
pilot participant did not complete the TSAS scenario due to technical problems 
with the TSAS tactors. 


Pre-Flioht Questionnaire Data 

Participant 

Number 

Age(yrs) 

Gender 

Dominant 

Hand 

Coior 

Biind 

Fiight 

Experience 

Fiight 

Time(Hrs) 

Primary 

Type 

M71300 

44 

M 

R 

N 

None 

None 

None 

F80800 

53 

M 

R 

N 

Yes 

4000 

Helicopter 

M91330 

33 

M 

L 

N 

Yes 

800 

Helicopter 

M161000 

29 

F 

R 

N 

Yes 

1250 

Helicopter 

M151400 

33 

M 

R 

N 

Yes 

1500 

Helicopter 

M151500 

31 

M 

R 

N 

Yes 

1900 

Helicopter 

F131300 

39 

M 

R 

N 

None 

None 

None 

F151300 

31 

M 

R 

N 

Yes 

95 

Fixed 

F191400 

34 

M 

L 

N 

None 

None 

None 

Average Age, 
Hours or 

Count 

36.33 

8 

7 

0 

6 

1590.83 

5 

Median Age 

33.00 





1375.00 



Table 4. Pre-Flight Questionnaire Baseline Data 


B. MEASURES OF PERFORMANCE 

Two measures of performance were collected using the X-Plane flight 
data from the six Participants completing both scenarios. The first measure 
recorded the increase in flight time the TSAS afforded the pilot to react and keep 
the aircraft airborne following a blinding scenario. To calculate the time of each 
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flight, researcher divided recorded frame units by the data collection rate of 15 
frames per second. Second, landing zone accuracy comparison with and without 
the TSAS measured navigation benefit of wearing the TSAS to reach the landing 
zone after a blinding scenario. 

To calculate a paired t-test and paired t-interval, the researcher checked 
the conditions and assumptions about the data. The measures of performance 
were on the same Participant before and after use of the TSAS. Behavior by 
each participant was independent of the others and observed differences were 
independent. The measured values were random for each individual and the 
distribution of the differences tended toward normality. Six of the nine 
Participants successfully completed the events with and without the TSAS. 
Added flight time with the TSAS was evident using a paired t-test with a t= 2.695 
with five degrees of freedom and p = .043. Results using a difference in time 
before landing or impact shown in Table 4 support the flight time benefit of the 
TSAS to allow the pilot from nine to 46 seconds more time for vision to return and 
continue flight. 
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Flight Time Before Landing/Impact Using TSAS Compared to 
Without TSAS 

Participant 

Time With TSAS 
(sec) 

Time Without TSAS 
(sec) 

Difference 

(sec) 

M7130 0 

74.1 

20.0 

54.1 

M151500 

57.8 

50.6 

7.2 

M91330 

67.9 

62.1 

5.8 

F191400 

97.3 

95.5 

1.8 

M161000 

89.9 

42.1 

47.9 

M151400 

88.1 

49.1 

38.9 



Mean Difference 

25.9 



Standard 

Deviation 

21.5 



Standard Error 

8.8 



df of 5 at 95% 

2.015 



Margin of Error ts 

17.3 



95% confidence 

25.9 + /-17.3 



or interval of 

(8.6,46.2) 

sec 

Results indicate with 95% confident that for pilots wearing 
the TSAS during an approach flight after a loss of vision on 
average have between nine and 46 seconds more time in flight 
than pilots without the TSAS. 


Table 5. Flight Time Gained Using TSAS 

Paired t-test using the TSAS for navigation was not statistically significant 
with a t = -1.33, p = 0.225. Graphical displays of the TSAS approach showed 
four out of six Participants responded to tactor cues and continued landing zone 
navigation. Figures 19-24 show the without-TSAS plot line first followed by the 
with TSAS plot. The zero axes indicate the Participant flew directly over the 
landing zone. A negative value indicated participant flew past the landing zone. 
The smaller the difference, the closer the participant flew toward the landing 
zone. 
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M71300 Landing Zone Difference Before Landing/Impact Without and With TSAS 



Figure 19. Participant M71300 Landing Zone Difference Plot 


Participant M71300 in Figure 19 and participant M161000 in Figure 20 
crashed quickly without the TSAS however with the TSAS cueing flew closer to 
the landing zone. 
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Latitude Difference Degrees North 


M16100 Landing Zone Difference Before Landing/Impact Without and With TSAS 
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Figure 20. Participant M161000 Landing Zone Difference Plot 



M151500 Landing Zone Difference Before Landing/Impact Without and With 

TSAS 
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Figure 21. Participant 151500 Landing Zone Difference Plot 
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Participant M151500 in Figure 21 using the TSAS maintained a direct path 
toward the landing zone using the TSAS. 


M9-1330 Landing Zone Difference Before Landing/Impact Without and With 

TSAS 
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Figure 22. Participant M9-1330 Landing Zone Difference Plot 


Figure 22 M9-1330 (see Figure 22) followed a direct path but ended up 
diverting. Participant indicated experiencing confusion and reverse cueing on 
post flight questionnaire. Participant Ml 51400 in Figure 23 with TSAS navigated 
the best of all completing the tasks. 
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M151400 Landing Zone Difference Before Landing/Impact Without and With 

TSAS 
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Figure 23. Participant M151400 Landing Zone Difference Plot 


F191400 Landing Zone Difference Before Landing/Impact Without and With 
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Figure 24. Participant F191400 Landing Zone Difference Plot 
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Table 5 containing paired interval test data reveals a positive trend by 
Participants to navigate closer to the landing zone with the TSAS. Decimal 
degree values were converted to feet using 6076 feet per one-degree latitude. 


Flight Deviation From Landing Zone Using TSAS Compared to Without 
TSAS 

Participant 

Deviation 

With TSAS 
(deg) 

Deviation Without 
TSAS (deg) 

Difference (deg) 

M71300 

-0.00319 

0.05239 

-0.05558 

M151500 

0.01197 

0.01310 

-0.00113 

M91330 

0.01853 

0.01358 

0.00495 

F191400 

0.01070 

0.00364 

0.00706 

M161000 

0.01799 

0.04645 

-0.02846 

M151400 

0.00110 

0.01008 

-0.00898 



Mean Difference 

-0.01369 



Standard 

Deviation 

0.02421 



Standard Error 

0.00988 



df of 5 at 95% 

2.015 



Margin of Error 
ts 

0.01992 



95% confidence 

(-0.01369 +/-0.1992) 



or interval of 

(-0.03361 , 0.00623) 

degrees 

Negative differences indicated the TSAS allowed participant to get 
closer to the landing zone. Results indicate with 95% confident 

that for pilots wearing the TSAS during an approach flight after a 
loss of vision on average were up to 0.0331 degrees (203 feet) 
closer or 0.00623 degrees (37 feet) farther from the landing zone 
than pilots without the TSAS. 


Table 6. Landing Zone Deviation Table 


Flight Profiles of the Participants provided a visual indication of how well 
the participant controlled decent during the approach using only the TSAS to 
provide cues. Figure 25 is a plot of an ideal approach assuming immediate 
transition time to the TSAS after a blinding event and immediate response to 
cues. 
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Ideal Altitude Profile 



Time Frame 


Figure 25. Ideal Approach Altitude Flight Profile 

Figures 26-31 illustrate altitude approach profiles with and without the 
TSAS. The altitude plots revealed a tendency of the Participants to pull back on 
the stick and gain altitude immediately after the blinding event. This action was 
useful to remain clear of obstacles, however, airspeed dropped off and the 
aircraft rolled and descended because of the simulated reduction of aerodynamic 
lift necessary keep the aircraft airborne. Participants M71300 in Figure 28, 
M91300 in Figure 29, and Ml51400 in Figure 31 with the TSAS responded 
initially to the cues but lost control because of disorientation or reversed the 
inputs for the cues during the approach resulting disorientation and the aircraft 
crashed. 
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M16100 Flight Profile With and Without TSAS 
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Figure 26. Participant M161000 Altitude Flight Profile 


F191400 Altitude Profile With and Without TSAS 



Figure 27. Participant F191400 Altitude Flight Profile 
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Altitude Feet MSL Altitude Feet 


M71300 Without and With TSAS 
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Figure 28. Participant I\/I71300 Altitude Flight Profile 
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M91330 Approach Without and With TSAS 
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Figure 29. Participant M91330 Altitude Flight Profile 
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Figure 29 and Figure 30 show instances in which disorientation occurred 
from which recovery or a safe landing was not possible for the time remaining 
and proximity to terrain. 
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M151500 Approach Without and With TSAS 
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Figure 30. Participant M151500 Altitude Flight Profile 


M151400 Approach Without and With TSAS 



Figure 31. Participant M151400 Altitude Flight Profile 
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c. 


POST-FLIGHT QUESTIONNAIRE 


After Phase 2, Participants answered a Post-Flight questionnaire (see 
Appendix C) to provide individual feedback. The purpose of the questionnaire 
was to record how each participant felt about the experiment, both with and 
without the vest, as well as the perceived usefulness of the TSAS. The 
questionnaire provided a quick check and immediate feedback concerning 
perceived self-improvement provided by using the TSAS. The questionnaire 
used a Likert scale with a score of one recording no improvement to a score of 
10 recording a significant level of improvement. Besides improvement, the 
questionnaire allowed Participants to indicate a level of perceived ability to 
maintain a heading and to keep the aircraft out of unusual attitudes of pitch and 
roll. The questionnaire period provided an opportunity for the researcher to 
observe for simulator sickness to ensure that no symptoms existed prior to the 
participant leaving the experiment. None of the Participants experienced or 
showed signs of simulator sickness. 

Non-pilot Participants required more time for the simulator familiarization 
portion but according to the questionnaire results, they understood the tactor 
input cues the same as pilots. Participants indicated haptic feedback was an 
interesting concept with current pilot culture receptive if given time to adjust. One 
pilot wanted to integrate the TSAS into the training for next generation pilots. 
Another suggested the tactors provided good input, but desired a variable input 
with frequency and strength when roll or pitch became extreme. Participants 
suggested adding voice for altitude and power cues to improve performance. 
Pilot Participants indicated the addition of a radar altimeter cue would assist 
them. One participant recommended a better design of cyclic pitch input 
because of possible confusion with roll inputs. Two Participants requested a 
voice headset capability to supply altitude callouts from ground radar altimeter to 
assist landing the aircraft. One pilot participant thought the inputs were too quick 
to register the input and react. The Participant attributed the comment to the 
training and familiarization time used for this evaluation. With a longer training 
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period, the cognitive load decreased from familiarity with input and response 
recognition. Table 6 displays a summary of each Participant’s response to the 
Likert scale questions. An impact prior to the airfield defined a crash during the 
study recorded in Table 7. 


Post-Flight Questionnaire 


With 




Maintained 

Returned 



TSAS 

Maintained 

Maintained 

Understood 

Levei 

to Levei 

TSAS 


First 

Hover 

Course 

inputs 

Attitude 

Attitude 

Heiped 

M71300 

X 

4 

4 

3 

4 

4 

4 

F80800 








M91330 


5 

5 

8 

7 

6 

8 

M161000 


5 

3 

7 

5 

5 

9 

M151400 


4 

8 

6 

9 

6 

10 

M151500 

X 

5 

6 

6 

7 

7 

10 

F131300 








F151300 








F191400 

X 

10 

8 

8 

10 

10 

9 

Mean 


5.5 

5.7 

6.3 

7.0 

6.3 

8.3 

SD 


0.49 

1.7 

1.7 

1.7 

1.0 

2.2 


Table 7. Post-Flight Questionnaire Key Responses 


Participant Results In Virtual Flight 
Environment 


Completed 

Tasks 

Crashed 
With TSAS 

Crashed 
Without TSAS 

M71300 

Yes 

Yes 

Yes 

F80800 

Yes 

NA 

Yes 

M91330 

Yes 

Yes 

Yes 

M161000 

Yes 

No 

Yes 

M151400 

Yes 

Yes 

Yes 

M151500 

Yes 

No 

Yes 

F131300 

Yes 

NA 

Yes 

F151300 

Yes 

NA 

Yes 

F191400 

Yes 

No 

Yes 

Count 

9 

3 

9 


Participants did not complete both parts 


Table 8. Crash Results using Prototype TSAS 
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Questionnaire data gave a quick indication of the benefit of the TSAS with 
a mean help score of 8.3 out of 10 recorded. The Participant responses 
confirmed the improvement recorded by comparing airfield latitude coordinates. 
Altitude plots revealed a trend of the pilots to pull back on cyclic and gain altitude 
after the simulated vision loss and then adjust to the inputs from the TSAS. 
Graphs depicting a rapid decrease in altitude over a short time frame indicated 
the time when the Participant entered an unrecoverable attitude and resulted in a 
simulator crash. 

The documented increase in time between eight seconds and 46 seconds 
support the advantage of the TSAS compared to without the TSAS. The altitude 
figures showed a characteristic distinct gain trend in altitude of the participant 
after the simulated loss of vision and the visually noted difference in time before 
the participant entered into an unrecoverable attitude. The second spike in 
altitude for the participant without the TSAS was the result of participant hearing 
the audio sound from the speakers corresponding to increased airspeed and 
engine speed sounds followed by a last ditch pullback on the cyclic by participant 
to gain altitude. Such a move from the pilot resulted in disorientation and 
ultimately a crash. Clear dots indicated speed before impact was excessive. 
Improved accuracy navigating to the landing zone demonstrated the advantage 
of the TSAS to guide the aircraft and pilot over the landing zone. Such proximity 
is sufficient for simulated tower personnel or ground personnel to provide 
potential voice communications and safety talk the pilot down onto the landing 
zone. 
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V. CONCLUSION AND RECOMMENDATIONS 


A. CONCLUSION 

The prototype Tactile Situational Awareness System provided sufficient 
haptic feedback to keep three out of six Participants oriented toward the landing 
field without vision during an approach. Vision can be augmented using haptics 
and the TSAS is a candidate solution. The results of this simulator study are 
consistent with the findings by Cheung et al (2004) who reported the TSAS 
provided appropriate cues for Participants to maintain orientation. The increase 
in time provided by using the TSAS to augment vision is supportive of the 
findings of Nakagawara et al (2003) who reported visual acuity returned after an 
intentional blinding episode. Validation of time gained was evident in the present 
results by observing the paired time difference between Participants with and 
without the TSAS until the aircraft reached the airport. 

Trend characteristics shown in the altitude plots suggested a tendency for 
the pilots to gain altitude to avoid obstacles on the ground. Without the TSAS 
providing feedback, all Participants eventually crashed. Critical flight time 
advantage of 8.6 to 46 seconds, noted in Table 4, allowed the Participant to 
maintain situational awareness because of the quick transition from vision to 
cues from the TSAS. In a loss or reduced vision situation, contributing to the 
pilot’s vision recovery time was important. Navigation using heading cues from 
the TSAS was successful with three Participants landing on average within 83 
feet of the landing zone as noted in Table 5. Participants demonstrated they 
correctly interpreted cues and continued flight toward the runway sufficient to 
reach the landing zone. Three Participants responding to the TSAS vest 
eventually crashed; however, three responded to cues and landed as noted in 
Table 8 adding hope to the value added from the TSAS to reduce Class A 
mishaps. 

Participant feedback provided additional information. Participant M161000 
said that the two back tactor signals were confusing for mental mapping during 
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the training phase but learned where to move the controls by the actual test or 
second phase. Results from the Questionnaire comments noted two negative 
training consequences -- reversal of the two back inputs and applying reverse 
flight control input. With more research and added voice feedback, it may be 
possible to prevent undesired or reverse control input. As a training tool, 
Participants adapted to TSAS cues easily. Simulator training with the TSAS can 
provide a tactile training aid for instructors to correct common student pilot errors 
without physical hands on methods. Applications for the TSAS during brown out, 
white out, and laser conditions have excellent potential as a haptic instrument to 
maintain orientation and reduce or prevent low visibility related Class A mishaps. 

Observed learning by Participants during the familiarization phase 
demonstrated that use of the TSAS prototype was easy to learn. The 
familiarization and training phase allowed the Participants to map the tactor 
inputs to flight control inputs. Participant M151500 felt the TSAS was an 
interesting concept and admitted prior training instinctively triggered a desire to 
continue scanning inside for the instruments after vision was lost. Using the 
TSAS allowed for an alternative method to support vision, which remains the 
primary method for pilots to obtain safety of flight information. Transfer of the 
TSAS prototype and integrating it with the computer system of a fully functioning 
aircraft is promising given that more aircraft control inputs are fly-by-wire. 

Confirmation of the need for the development of a situational awareness 
or disorientation warning device such as the TSAS as an aid for pilots appeared 
in a message in October 2008 after the Program Objective Memorandum (POM- 
12) Aircrew Systems (ACS) Enabler Naval Aviation Requirements Group (NARG) 
meeting. One of the results of the meeting included: 

Spatial Disorientation Warning Device - Spatial disorientation is a 
major human factor that contributes to present day mishaps. 
Recommend the development of a device that will provide sensory 
stimulation in response to a spatial disorientation event that will 
provide sensory stimulation in response to a spatial disorientating 
event. 
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B. RECOMMENDATIONS 


Continue testing changes in technology to improve human machine 
interoperability. The first is to determine how small a degree change in heading 
the average person can control using TSAS. Second, investigate the level of 
heading control in a blind condition with the TSAS combined with the benefit of 
voice heading cues to compliment the tactile cues. For altitude control, a third 
recommendation is to incorporate an additional input tied to the radar altimeter to 
set and monitor an altitude and have tactor cues alert the pilot if above or below 
a desired altitude. An example of an altitude control situation is night flight over 
water or over terrain to avoid power lines or obstacles. Applying the TSAS with 
Autonomous Aerial Vehicle (AAV) ground operators is promising to help lower 
the number lost because of not knowing the correct orientation during flight. The 
use of X-Plane allows repeating the experiment model and simulating a gradual 
restoration of scenery after thirty seconds and monitor recover techniques from 
loss of sight as mentioned by the FAA laser test study (Nakagawara et al (2003). 
This thesis hopes to contribute to a solution to the laser blindness threat to 
military, commercial and general aviation. If advances in technology continue, 
the aviation and unmanned vehicle community stand to gain if TSAS procedures 
are approved and placed in NATOPS manuals and FAA aeronautical manuals. 
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APPENDIX A: CODE FILES 


File: tactor.py 
Language: Python 
Compiler: Eclipse 3.2 

Developed by J. S. Brown and Randy Jones 


import threading, time 


EAI_COMMAND_OK = '\x80' 
EAI_DEFAULT_FREQUENCY =258 # 

MAX_ON_TIME =1000 # 

TACTOR_SHOT_l = chr(O) # 

device 1 

TACTOR_SHOT_2 = chr(l) # 

TACTOR_SHOT_3 = chr(2) 

TACTOR_SHOT_4 = chr(3) 

TACTOR_LEFT_SHOULDER = chr(4) 
TACTOR_RIGHT_SHOULDER = chr(5) 
TACTOR_DISTANCE_l = chr(6) 
TACTOR_DISTANCE_2 = chr(7) 
TACTOR_DISTANCE_3 = chr(8) 
TACTOR_DISTANCE_4 = chr(9) 


Tactor Frequency in Hz 
Time in milliseconds 
0 corresponds to 


class commander(threading.Thread): 

def _init_(self, serialPort): 

threading.Thread._init_(self) 

self.setDaemon(True) # Daemon threads stop with 
main thread 

self.serial_port = serialPort 
self.serial_port.write('\x21') 
self.stop = False 

self.on = [False, False, False, False, False, 
False, False, False, False, False] 
self.seconds = 0 

def TurnOn(self, tactor, time): 
if( time > MAX_ON_TIME ): 
time = MAX_ON_TIME 

self.serial_port.write('\xl1' + tactor + 
chr(time/10)) 


def TurnLeft(self) : 

global TACTOR_LEFT_SHOULDER 

self.TurnOn(TACTOR_LEFT_SHOULDER, 50) 
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def TurnRight(self): 

global TACTOR_RIGHT_SHOULDER 

self.TurnOn(TACTOR_RIGHT_SHOULDER, 50) 

def end(self): 

self.serial_port.close() 
self.stop = True 

def run(self) : 

while not self.stop : 
time.sleep (2.0) 
if (self.stop): 

break 
1 = 0 ; 

for turn_on in self.on: 
if turn_on: 

print 1, "on" 

self.TurnOn(chr (1), 100) 

1 = 1 + 1 

self.seconds = self.seconds + 2 
print self.seconds, "seconds" 
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File: udp.py 
Language: Python 
Compiler: Eclipse 3.2 

Developed by James S. Brown and Randy Jones 

from socket import * 
import threading, struct 

from geo_helper import calculate_distance_and_bearing 
ValueMap = dict() 

### This thread receives UDP data from X-Plane output ### 

class listener(threading.Thread) : 

def _init_(self, ioPort, factor, uidata): 

threading.Thread._init_(self) 

self.factor = factor 
self.uidata = uidata 

self.SOCKET=socket(AF_INET, SOCK_DGRAM) 
self.SOCKET.bind( ( ' ',int (ioPort))) 
self.setDaemon(True) 

### Daemon threads stop with main thread ### 
self.stop = False 
self.prev_rudder = 0.0 
self.heading = None 
self.prevHeading = None 
self.off = 0 
self.lat = None 
self.long = None 

### Output screen data display X-Plane feed received ### 
def dumpData(self, data): 

1 = 0 

for c in data: 
i = i + 1 

print '%02X' % ord(c), 
if (1 % 20) == 0: 

print "" 
print "" 

def end(self): 

self.stop = True 

self.SOCKET.shutdown(SHUT_RDWR) 
self.SOCKET.close () 
self.tactor.end 0 
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def RollControl(self, roll): 

THRESHOLD =7.0 

### Max roll degrees before cue received by operator ### 
### print "roll control", roll ### 
if (roll > THRESHOLD): 

self.factor.on[0] = True 
self.factor.on[1] = Ealse 
elif (roll < -THRESHOLD): 

self.factor.on [ 0] = False 
self.factor.on[1] = True 
else: 

self.factor.on [ 0] = False 
self.factor.on[1] = False 

def PitchControl(self, pitch): 

### print "roll control", pitch ### 
if (pitch > 20): 

self.factor.on[2] = True 
self.factor.on[3] = False 
elif (pitch < -8): 

self.factor.on[2] = False 
self.factor.on[3] = True 
else: 

self.factor.on [2] = False 
self.factor.on[3] = False 

def AntiTorq(self, hdg): 

THRESHOLD =4.0 

### tolerance plus and minus 4 deg hdg before cue ### 
if self.heading == None: 
self.heading = hdg 
self.prevHeading = hdg 
self.off = 0 
else: 

### adjust for 359/0 transition ### 
if (hdg - self.prevHeading) > 180: 

self.off = self.off + 360 
elif (hdg - self.prevHeading) < -180: 

self.off = self.off - 360 
# heading trigger calculation 
dif = self.heading - hdg + self.off 
print self.heading, hdg, self.off, dif, 
self.prevHeading 

if dif > THRESHOLD: 

self.factor.on[8] = False 
self.factor.on[9] = True 
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elif dif < (-1*THRESH0LD): 
self.tactor.on [ 8] = True 
self.factor.on[9] = False 
else: 

self.tactor.on[8] = False 
self.tactor.on [ 9] = False 
self.prevHeading = hdg 

def procData(self, data): 

BLEN = 6 

XLEN =36 

global ValueMap 

qty = (len(data) - 6) / XLEN 

for 1 in xrange(qty): 

id = data[BLEN+(XLEN*i):BLEN+(XLEN*i)+4] 
id = struct.unpack ('1', id) 
id = id[0] 

substr = data[BLEN+4+(XLEN*i):BLEN+4+(XLEN*i)+32] 
values = struct.unpack ('8f', substr) 

ValueMap[id] = values 

### Look for hdg change from X-Plane Index 16 ### 

if self.uidata.AntiTorqOn: 

self.AntiTorq(ValueMap[16][2]) 
else: 

self.heading = None 
self.tactor.on[8] = False 
self.tactor.on[9] = False 

### Look for Roll value from Level horizon ### 
if self.uidata.RollControlOn: 

self.RollControl(ValueMap[16] [1] ) 
else: 

self.tactor.on[0] = False 
self.tactor.on[1] = False 

### Look for Pitch value from Level horizon ### 
if self.uidata.PitchControlOn: 

self.PitchControl(ValueMap[16][0]) 
else: 

self.tactor.on[2] = False 
self.tactor.on[3] = False 

def run(self) : 

while not self.stop : # Receive in daemon thread 
data, address = self.SOCKET.recvfrom(1024) 
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if data and data[0:4] == "DATA": 
self.procData(data) 
self.SOCKET.close () 
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File: Xplane.py 
Language: Python 
Compiler: Eclipse 3.2 

## This class allows user to activate Tactor Suit ## 
import wx, wx.lib.newevent 

import sys, os.path, threading, time, serial 
import udp, tactor 

tactor_task = None 
udp_task = None 

# Initial Status Assignments # 

class UIdata: 

AntiTorqOn = False 
RollControlOn = False 
PitchControlOn = False 

uidata = UIdata () 

# Create GUI interface and labels # 

class MainFrame(wx.Frame): 

def _init_(self, parent=None, id=-l, 

pos=wx.DefaultPosition, 

title="NPS - X-Plane Tactor Program"): 

wx.Frame._init_(self, parent, id, title, pos, 

size=(400, 300)) 

panel = wx.Panel ( self, -1 ) 
vs = wx.BoxSizer( wx.VERTICAL ) 
hs = wx.BoxSizer( wx.HORIZONTAL ) 
vs.Add(hs, I, wx.ALIGN_CENTREIwx.ALL, 5 ) 

# X-Plane interface 

boxl_title = wx.StaticBox( panel, -1,"X-Plane Interface" ) 
boxi = wx.StaticBoxSizer ( boxI_title, wx.VERTICAL ) 
gridi = wx.ElexGridSizer( 0, 2, 0, 0 ) 
labelll = wx.StaticText (panel, -1, "IP Port") 
gridi.Add( labelll, 0, 

wx.ALIGN_CENTREIwx.LEETIwx.RIGHTIwx.TOP, 5 ) 
self.textll = wx.TextCtrl( panel, -1, "4000" , 
style=wx.TE_RIGHT) 
gridi.Add( self.textll, 0, 

wx.ALIGN_CENTREIwx.LEETIwx.RIGHTIwx.TOP, 5 ) 
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boxl.Add( gridl, 0, wx.ALIGN_CENTRE|wx.ALL, 5 ) 
hs.Add( boxl, 1, wx.ALIGN_CENTREIwx.ALL, 5 ) 

# Tactor interface 

box2_title = wx.StaticBox ( panel, -1, "Tactor Vest 
Interface" ) 

box2 = wx.StaticBoxSizer ( box2_title, wx.VERTICAL ) 
grid2 = wx.ElexGridSizer( 0, 2, 0, 0 ) 
label2I = wx.StaticText (panel, -1, "Serial Port") 
grid2.Add( label21, 0, 

wx.ALIGN_CENTREIwx.LEETIwx.RIGHTIwx.TOP, 5 ) 
ports = ["COMl", "COM2", "COM3", "COM4", "COM5", 
"COM6", "COM7", "COM8", "COM9"] 

self.combo21 = wx.ComboBox(panel, -1, "COMl", (90, 
50), (95, -1),ports, wx.CB_DROPDOWN|wx.CB_READONLY) 

grid2.Add( self.combo21, 0, 

wx.ALIGN_CENTREIwx.LEETIwx.RIGHTIwx.TOP, 5 ) 
box2.Add( grid2, 0, wx.ALIGN_CENTRE|wx.ALL, 5 ) 
hs.Add( box2, 1, wx.ALIGN_CENTRE|wx.ALL, 5 ) 

# test 

box3_title = wx.StaticBox( panel, -1, "Vest Test" ) 
box3 = wx.StaticBoxSizer ( box3_title, wx.VERTICAL ) 
grids = wx.ElexGridSizer( 0, 3, 0, 0 ) 

#self.combo22 = wx.ComboBox(panel, -1, "1", (70, 

50) , 

# (95, -1), xrange(ll), 

wx.CB_DROPDOWN|wx.CB_READONLY) 

time = '2', '3', '4', '5', '6', '7', '8', 

'9', '10'] 

self.combo22 = wx.ComboBox(panel, -1, (100, 

50) , 

(95, -1), time) 
self.combo22.SetSelection(0) 

stuff2 = ["shotl", "shot2", "shotS", "shot4", "left 
shoulder","right shoulder", "distancel", "distance2", 
"distances", "distance!"] 

# new stuff 

stuff = ["UpperLeft 1", "UpperRight 2", 
"UpperRightBack 3", "UpperLeftBack 4", "LowerLeftEront 5", 
"LowerRightBack 6", "LowerLeft 7", "LowerRight 8", "LeftLeg 
9", "RightLeg 10"] 

self.combo23 = wx.ComboBox(panel, -1, "UpperLeft 1", (100, 

50), (95, -1), stuff, wx.CB_DROPDOWNIwx.CB_READONLY) 
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grids.Add( self.combo23, 0, 
wx.ALIGN_CENTREIwx.LEFTIwx.RIGHTIwx.TOP, 5 ) 
grids.Add( self.combo22, 0, 
wx.ALIGN_CENTREIwx.LEFTIwx.RIGHTIwx.TOP, 5 ) 

self.tb = wx.Button (panel, -1, "test") 
grids.Add (self.tb, 0, wx.ALIGN_CENTRE|wx.ALL, 5 ) 
self.Bind(wx.EVT_BUTTON, self.test, self.tb) 
boxS.Add( grids, 0, wx.ALIGN_CENTRE|wx.ALL, 5 ) 
vs.Add( boxS, 1, wx.ALIGN_CENTREIwx.ALL, 5 ) 

# Anti-Torque button 

self.ab = wx.Button(panel, -1, "Anti-Torq (OFF)") 
vs.Add (self.ab, 0, wx.ALIGN_CENTRE|wx.ALL, 5 ) 
self.Bind(wx.EVT_BUTTON, self.antiTorq, self.ab) 

# roll control button 

self.rc = wx.Button (panel, -1, "Roll-Control 
(OFF)") 

vs.Add (self.rc, 0, wx.ALIGN_CENTRE|wx.ALL, 5 ) 
self.Bind(wx.EVT_BUTTON, self.RollControl, self.rc) 

# roll control button 

self.pc = wx.Button(panel, -1, "Pitch-Control 
(OFF)") 

vs.Add (self.pc, 0, wx.ALIGN_CENTRE|wx.ALL, 5 ) 
self.Bind(wx.EVT_BUTTON, self.PitchControl, 
self.pc) 

# Start button 

self.sb = wx.Button(panel, -1, "START") 

vs.Add (self.sb, 0, wx.ALIGN_CENTRE|wx.ALL, 5 ) 

self.Bind(wx.EVT_BUTTON, self.start, self.sb) 

# Roll, Pitch, and Anti-Torque label 

panel.SetSizer ( vs ) 
vs.Fit ( panel ) 
self.panel = panel 

def RollControl(self, event): 
global uidata 

uidata.RollControlOn = not uidata.RollControlOn 
if uidata.RollControlOn: 

self.rc.SetLabel("Roll-Control (ON)") 
else: 

self.rc.SetLabel("Roll-Control (OFF)") 
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def PitchControl(self, event): 
global uidata 

uidata.PitchControlOn = not uidata.PitchControlOn 
if uidata.PitchControlOn: 

self.pc.SetLabel("Pitch-Control (ON)") 
else: 

self.pc.SetLabel("Pitch-Control (OFF)") 

def antiTorq(self, event): 
global uidata 

uidata.AntiTorqOn = not uidata.AntiTorqOn 
if uidata.AntITorqOn: 

self.ab.SetLabel("Anti-Torq (ON)") 
else: 

self.ab.SetLabel("Anti-Torq (OFF)") 


def 


test. ' , 


test(self, event) : 
if tactor_task == None: 

dig = wx.MessageDialog(self, 'Stopped, start to 

'Error', wx.OK | wx.ICON_INFORMATION) 
dig.ShowModal() 
dig.Destroy () 
return 

sensor = chr(self.combo23.GetSelection()) 
print self.combo22.GetSelection() 
time = self.combo22.GetSelection() * 100 
tactor_task.TurnOn(sensor, time) 


def start(self, event): 


# set up serial port 

global tactor_task, udp_task, uidata 
sel = self.combo21.GetSelection() 
serial_port = None 
if udp_task != None: 
udp_task.end() 

try: 

serial_port = serial.Serial (sel) 
except: 

print sys.exc_info()[0] 
if serial_port == None: 

s = self.combo21.GetValue() 

dig = wx.MessageDialog(self, 'Could not 

intialize ' + s + ' . ' , 

'Error', wx.OK | wx.ICON_INEORMATION) 
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dig.ShowModal() 
dig.Destroy () 
else: 

self.sb.SetLabel("STOP") 

self.Bind(wx.EVT_BUTTON, self.stop, self.sb) 
udp_port = self.text11.GetValue() 
tactor_task = tactor.commander(serial_port) 
tactor_task.start() 

udp_task = udp.listener(udp_port, tactor_task, 
uidata) 

udp_task.start() 

def stop(self, event): 

global tactor_task, udp_task 

udp_task.end() 

udp_task = None 

tactor_task.end() 

tactor_task = None 

self.sb.SetLabel("START" ) 

self.Bind(wx.EVT_BUTTON, self.start, self.sb) 

class App(wx.App): 

def Oninit(self): 

self, frame = MainErameO 

self.frame.Show() 

self.SetTopWindow(self.frame) 

self.frame.Centre() 

return True 

def getDir(): 

if os.path.isdir(sys.path[0]): 

dir = sys.path[0] 
else: 

dir = os.path.dirname(sys.path[0]) 
return dir + ' / ' 

def main(): 

global app 

app = App(O) # call with 0 to get debug message to 

sysout 

app.MainLoop () 

if _name_ == '_main_' : 

main () 
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APPENDIX B: PRE-FLIGHT QUESTIONNAIRE 


Q1: Do you play computer or flight simulator video games? 

YesQ or NoD 

Q2: How many hours a week do you play? 

< 5 hrs O or > 5 hrs O NA O 

Q3: Are you susceptible to motion sickness or get sick from video games? 

YesO NoO 
Q3a: If yes, please explain. 

Q4: How old are you? _years 

Q5: Please mark your sex. Male D or Female □ 

Q6: What is your dominant writing hand? 

Right Hand O or Left Hand Q 
Q7: Are you colorblind? 

Yes □ or No □ 

Q7 a; If color blind, which color or colors? 

RedQ, OrangeQ, Yellow^, Green^, BlueQ, PurpleQ 
Other_ 

Q8; Do you have any flying experience in a helicopter, airplane or glider? 
Yes □ or No □ 

Q8 a: If yes, check type and fill in how many hours flight time? 


Helicopter 

n 

Hours Flight Time 

Airplane 

□ 

Hours Flight Time 

Glider 

□ 

Hours Flight Time 
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APPENDIX C: POST- FLIGHT QUESTIONNAIRE 


Instructions: Please answer the following questions. One(1) represents 
lowest score and ten(10) best score. 

1. Please check if you conducted flight with TSAS first or without TSAS 
first? 

a. TSAS First □ 

b. Without TSAS □ 

2. Rate how well you were able to turn left and right using the input 
device? 

in ?□ sD 4n sD en /□ sn qd ion 

3. How well were you able to maintain a hover with TSAS? 

in ?□ 30 40 sD en /□ 80 90 ion 

4. How well were you able to maintain heading with TSAS? 

in ?□ 3n 4n sn en sn gn ion 

5. How well were you able to understand input signals from TSAS? 

in ?□ 30 40 sD en /□ 80 90 ion 

6. How well were you able to remain wings level or on a level horizon with 
input device? 

in ?□ 3n 4n sn en /□ 8n qd ion 

7. How well were you able to return to wings level or to a level horizon with 
input device? 

in ?□ 3 n 4 n sn en /□ 8 n gn ion 

8. Your role in the scene was to either maintain a hover or follow a 

heading signal to an airfield. How well to you feel the TSAS helped 

compared to without the TSAS? 
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in 2n sD 4n sn en /□ sn gn ion 

Comment_ 

9. Did you feel any type of simulator sickness during the test? 

Yes □ NoQ 

If yes, rate level of simulator sickness and duration in minutes? 

in ?□ sD 4n sD en /□ sn gn ion 

Duration of feeling simulator induced sickness_minutes. 

10. Please check if you were able to accomplish the following initial tasks: 

Task 1. Were you able to maintain altitude? YesQ NoQ 

Task?. Did you crash the simulator without TSAS? Yes D No □ 

Task 2a. Did your crash with TSAS? Yes □ No O 

Task 3. Did you maintain the navigation radial to the airfield from 
(starting point)? Yes □ No.Q 

Task 4. Were you able to navigate back to course assigned? 

Yes □ No O 

11. Which Task did you omit during the study? 

in2n3n 4n5n n/ad 
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APPENDIX D: INSTITUTIONAL REVIEW BOARD APPROVAL 

LETTER 


Naval Postgraduate School 

Institutional Resiew Board (IRB) 


lt-Jia-07 


Fram: LT Brcal OUe. Ph.I> 

Tac Rewach Aualaai Ptofeaor Willum Bcclia 

Rcacarch Proteaor Michad McCaitey 
LdXtJnes S. Bromi 

S«t9«t YOUR PROJECT: EVALUATKJN OF TACTILE StTUATlON 
AWARENESS SUIT TO AUGMENT VISION AND IMPROVE 
SPATIAL ORIENTATXJN IN A VIRTUAL ENVIRONMENT 


1. The NFS IRB is pl c uad Ip ■ fane yoa dal Ifct NFS la*i miOTiM l R«vK«r 
Boefd ha n yrote d yoar pto j e ct (NPS IRBa NFS20OT00)l^ 

2. TheNPSIRBwuanaiMllyocntAf>dbyBUk<EOea26laly20Q2nd 
hu been re-cotified anhl )0 Math 2007 

3. This approve] N valid fur one yea fron Om dale Pleoe ohaii a oofyot 
all records and cociscnl ibrais to Ihc Rasaarch and Spo ns orod Proy 
Oflicc (Laun Ann Small, HalliM* Hall. Room 201B] a da oeachawn of 
this project. 

4. If your protocol changes at any time, you will need to resubout your 
pnqeci propoal to the NPS IRB 

Sinearely, 

Lt Brent Olde, Ph.D 

Chair 

NPS Inatitutional Review Board 
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